新增跨任务 AI 记忆系统 + 绕过微信 8.0.52+ 无障碍节点隐藏 - #8
Open
plzkeepaway-huaQ wants to merge 1 commit into
Open
Conversation
- New TaskMemory + RememberTool: Persists task learnings across sessions (MMKV-backed, 100 entries max), auto-injected into system prompt - New ScreenMemory: Caches UI element positions per app, reused across tasks (7-day expiry) — integrated into GetScreenInfoTool and FindNodeInfoTool - WeChat operation guide: Added chat-specific navigation guidance and safety rules (avoid Moments/Discover tab) - General operation rules: Page verification before actions, retry limits - Fix maxIterations Builder default (20 → 60) to match AgentConfig - Increase tool result truncation limit (300 → 2000 chars) for better AI context - Register RememberTool in ToolRegistry Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: 新增跨任务 AI 记忆系统 + 绕过微信 8.0.52+ 无障碍节点隐藏
概述
这个 PR 解决两个长期存在的问题,显著提升 ApkClaw 在微信场景下的自动化能力:
功能一:跨任务 AI 长期记忆系统
问题
之前 Agent 每次执行任务都是"失忆"状态 —— 上一轮学会了某个按钮的坐标,下一轮又要重新扫描。典型痛点:
解决方案
引入了两层记忆系统:
① 任务经验记忆(TaskMemory + RememberTool)
Agent 完成一个任务后,通过
remember(action="save", ...)保存关键步骤和经验教训。下次新任务开始时,相关经验自动注入到系统提示中。save/search/list三种操作② UI 元素位置缓存(ScreenMemory)
每次
get_screen_info扫描屏幕时,自动把 UI 元素的坐标、类名、可点击性缓存下来。下次再进同一页面,直接贴坐标给 Agent:com.tencent.mm、com.android.launcher等)GetScreenInfoTool扫描时自动录入,FindNodeInfoTool查找时优先查缓存效果
功能二:绕过微信 8.0.52+ 无障碍节点隐藏
问题
微信 8.0.52 版本开始,主动检测正在运行的无障碍服务。对于第三方服务(即非系统自带的服务),微信故意将 UI 节点树中的元素标记为不可见(
AccessibilityNodeInfo.isVisibleToUser() = false),导致:get_screen_info返回空白或残缺的屏幕内容find_node_info找不到任何元素社区的典型绕过方法(如反射修改 AccessibilityServiceInfo、Hook 系统 API)都因 Android 14+ 权限收紧而失效。
解决方案
采用双重策略:
① 伪装系统无障碍服务(SelectToSpeakService)
注册了一个完全空白的无障碍服务,其完整限定名伪装成 Google 系统服务:
关键细节:
onAccessibilityEvent()为空)AndroidManifest.xml中声明android:priority="10000",确保最先被系统注册ClawAccessibilityService执行,微信不感知② 宽松可见性模式(Lenient Visibility)
即使伪装成系统服务,微信仍可能在某些场景下把节点设为不可见。
ClawAccessibilityService实现了专门的应对:ANTI_A11Y_APPS— 维护反无障碍应用名单(微信、企业微信、支付宝、QQ)getScreenTreeSmart()— 检测到目标 App 在前台时,自动切换宽松模式buildNodeTreeLenient()— 获取 UI 树时忽略isVisibleToUser()标志,只要节点有文字、描述或交互属性就纳入结果findNodesByTextAntiObfuscated()— 三层查找策略:text和contentDescriptionviewId片段匹配(微信的 viewId 部分可读)效果
验证环境
其他改进
文件变更
注意事项
SelectToSpeak和ApkClaw,可能会显示两个Apkclaw的无障碍服务按钮,需同时开启